Papers with exact word matching
Sentence Mover’s Similarity: Automatic Evaluation for Multi-Sentence Texts (P19-1)
Copied to clipboard
| Challenge: | Existing automatic metrics for evaluating text are expensive and time-consuming. |
| Approach: | They propose automatic metrics that evaluate text in a continuous space using word and sentence embeddings. |
| Outcome: | The proposed method outperforms ROUGE on machine-generated summaries and human-authored essays on human-generated texts. |
Can LLMs Clarify? Investigation and Enhancement of Large Language Models on Argument Claim Optimization (2025.coling-main)
Copied to clipboard
| Challenge: | While Large Language Models (LLMs) have demonstrated proficiency in text rewriting tasks such as style transfer and query rewrite, their application to claim optimization remains unexplored. |
| Approach: | They propose to use a sliding window mechanism to evaluate the performance of large language models in claim clarification tasks under different settings. |
| Outcome: | The proposed model improves the performance of three LLMs on the claim clarification task under zero-shot, few-shot and supervised fine-tuning settings. |
Sentence Pair Embeddings Based Evaluation Metric for Abstractive and Extractive Summarization (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing evaluation metrics, such as ROUGE and BLEU, rely on exact word matching and fail to capture semantic similarity. |
| Approach: | They propose to use contextualized word or sentence embeddings to capture semantic similarity between sentences to evaluate text summarization methods. |
| Outcome: | The proposed evaluation metric shows that it performs faster than the current state-of-the-art on the SummEval dataset. |